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Abstract 
Attempts to scale up instructional interventions confront implementation challenges that mitigate 
their ultimate impact on teaching and learning. In this paper, we argue that learning about 
adaptation during the design and implementation phases of reform is critical to the development 
of interventions that can be implemented with integrity at scale. Through analysis of data 
generated during a mathematics instructional coaching initiative, we examine the adaptations 
coaches made to diverse relational and organizational contexts. Findings from two studies of 
adaptation illustrate the need to attend to the extent to which adaptations are consistent with the 
core features of a reform. Based on our findings, we posit a generalizable model that supports 
evidence-based mutual adaptation. 
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Learning from Adaptation to Support Instructional Improvement at Scale: 
Understanding Coach Adaptation in the TN Mathematics Coaching Project 

The history of education reform efforts is rife with examples of instructional innovations 
that falter when attempts are made to implement them at scale. As instructional innovations are 
taken from the specific contexts where they were developed, they confront variable systemic 
capacity to support implementation with integrity (LeMahieu, 2011; Penuel, Fishman, Cheng & 
Sabelli, 2011). Our work aims to contribute to research and practice related to the improvement 
of mathematics instruction at scale through instructional coaching. Instructional coaching is a 
promising approach to supporting teacher learning and instructional improvement. Well- 
designed coaching programs have demonstrated significant effects on teaching, and even student 
learning (Kraft, Blazer, & Hogan, 2018). 

However, attempts to spread and scale coaching programs have resulted in variable 
outcomes, in part because coaching is highly context dependent. Education reform efforts have 
promoted rapid growth in coaching positions in educational systems (Domina et al., 2015; 
Neufeld & Roper, 2003). For example, Desimone and Pak (2016) report that the Every Student 
Succeeds Act (ESSA) includes eleven references encouraging states and districts to utilize 
coaches to support improvement. As carriers of reform efforts, coaches operate as boundary 
spanners between federal, state, district, and school contexts; and their roles are often fluid and 
heavily dependent on contextual factors (Coburn & Russell, 2008; Domina et al., 2015; Kraft, 
Blazer & Hogan, 2018. Consequently, effective spread and scale of coaching interventions 
requires planning explicitly for the adaptation of a coaching approach into diverse local 


instructional systems. 
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We engaged in a collaborative project to develop, test, refine, and scale a model for 
mathematics instructional coaching: the TN + IFL Math Coaching Model.' The model provides: 
(1) a framework for evidence-based coaching practices that support teaching improvement, (2) 
an approach to training coaches to enact the coaching framework, and (3) guidance for schools, 
districts, and states on how to organize and support a coaching program. The model is designed 
to be a resource for schools and districts as they support the transition to teaching that is aligned 
with rigorous, college- and career-ready mathematics standards. As such, the coaching model is a 
tool to support mathematics instructional improvement at scale. By scale, in this context, we 
mean the use of our coaching model to support the uptake of standards-aligned mathematics 
instructional practices in classrooms throughout a state. In this sense, we evoke the notions of 
breadth and depth of implementation implicit in the scaling reform concept (Coburn, 2003). 

The goal of this paper is to describe what and how we learned about adaptation that 
supports a coaching model that can be deployed at scale. Specifically, we highlight what we have 
learned about coaches’ adaptation of the Model into diverse relational and organizational 
contexts, which represent two key points of variation that coaches confront in their work. 
Adaptation is evident in adjustments to the coaching practices that our coaches undertook as they 
worked with different teachers. By studying these adaptations, we learned about the relative 
importance of the coaching practices that define our model. Additionally, we present exploratory 
findings from our more explicit attempt to promote coach experimentation with adaptive 
integration utilizing the practitioner-led methods of improvement research. By adaptive 
integration we mean the use of systematic methods to learn how to implement a new practice or 
work process into a new context, one other than where it was developed (Hannan, et al., 2017; 


Tichnor-Wagner et al., 2018). In both studies, we aimed to learn about productive adaptation and 
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inform the spread of the model to additional coaches and contexts. Ultimately, we argue that 
explicit attention to adaptation in design projects is critical to the production of scalable 
interventions, and offer insights we learned about the adaptation process, including positing a 
generalizable model that supports evidence-based mutual adaptation. 
Conceptual Grounding 

Coaching to Support High Quality Mathematics Teaching and Learning 

Implementing college and career readiness-focused standards, such as those promoted by 
the Common Core State Standards, requires significant changes in typical teaching practice. For 
example, in mathematics, students are expected to not only develop fluency in the use of 
mathematical procedures, but also to develop deep conceptual understanding and the capacity to 
think, reason, and problem solve as they grapple with non-routine problems. The kinds of 
environments that support such learning are challenging to set up and, not surprisingly, 
somewhat atypical in U.S. classrooms (Hiebert & Stigler, 2004). Research tells us that changing 
instructional practice to become more rigorous and conceptually demanding is not a trivial 
undertaking (Correnti & Martinez, 2012; Stein et al., 2017; Thompson & Zeuli, 1999). Standard 
forms of professional development, such as workshops, are generally not sufficient to support 
this kind of instructional improvement (Birman, Desimone, Porter, & Garet, 2000; Sandholtz, 
2002; Kennedy, 2016; Yoon, Garet, Birman, & Jacobson, 2007). As a result, schools and 
districts have tried to orchestrate more intensive, job-embedded modes of professional 
development, such as coaching (Desimone & Pak, 2016). 

There is an emerging evidence base to suggest that instructional coaching is a promising 
intervention to support teaching improvement and student learning (Allen, Pianta, Gegory, 


Mikami, & Lun, 2011; Biancarosa, Bryk, & Dexter, 2010; Blazar & Kraft, 2015; Bryk, Gomez, 
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Grunow, & LeMahieu, 2015; Campbell & Malkus, 2011; Foster & Noyce, 2004; Garet, et al., 
2011; Killion, 2012; Mangin & Dunsmore, 2015; Matsumura et al., 2010; Matsumura, Garnier, 
& Spybrook, 2012; 2013; Neufeld & Roper, 2003; Neumann & Cunningham, 2009; Polly, 2012; 
Powell, Diamond, Burchinal, & Koehler, 2010; Sailors & Price, 2010). For example, a three-year 
randomized control study conducted by Campbell and Malkus (2011) found that coaches 
positively influenced elementary student mathematics achievement, particularly after coaches 
gained experience and skill through extensive professional development. However, studies of 
mathematics coaching have not always found significant effects on teaching or student learning 
(see for example Garet and colleagues, 2011). The mixed evidence for coaching can be partially 
attributed to the complexity of this professional practice and because coaching roles are 
structured and enacted in variable ways (Coburn & Woulfin, 2012; Mangin, 2009; Mangin & 
Dunsmore, 2015). This research suggests that there are important implementation challenges 
associated with the introduction of coaching programs in schools and districts. Learning from 
efforts to implement coaching through adaptive integration will help the field understand how 
and why different coaching models produce different results. 
TN + IFL Math Coaching Model 

Prior research on the promise and implementation challenges of coaching informed our 
work to design a model that could support its effective implementation in diverse local contexts. 
Our high-level representation of the model is presented in Figure 1. At the center is our coaching 
practice framework, which is comprised of three components: (1) a coach development 
framework that specifies our method for training coaches; (2) a coaching framework which 
specifies key coaching practices and routines; and (3) an ethos of continuous improvement which 


informs how coaches are trained to use disciplined inquiry cycles to adaptively integrate the 
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coaching model into their diverse local contexts. Other key components of the framework 
include our stance toward mathematics teaching and guidance regarding how schools, districts, 
and the state can support the implementation of the model. Here, we further elaborate on the 
coaching framework depicted at the center, because it is critical to understanding the adaptation 
process which represents the core features of the model. 

Coaching framework. Given the need to provide a sharper vision of coaching practice as 
noted in the research literature (Gibbons and Cobb, 2016), we developed a framework to 
establish a professional vision for what coaching practice should look like. Drawing on the 
practice-based experience of our Institute for Learning colleagues and the research-based 
insights about mathematics teaching and learning, our team identified three key coaching 
practices including: (1) deep and specific discussions of the instructional triangle; (2) 
establishing mathematics and pedagogical goals; and (3) evidence-based feedback. These 
practices are rooted in key tenets from the professional learning literature, such as the need for 
coaching interactions to be content specific (i.e., focused on mathematics teaching rather than 
general teaching strategies) (Desimone, Porter, Geret, Yoon, & Birman, 2002; Matsumura, 
Garnier, & Spybrook, 2012) and the power of substantive talk about content, student thinking, 
and pedagogy in supporting teachers use of high quality teaching practices (Coburn & Russell, 
2008; Lewis, Perry, & Hurd, 2009; Munter & Correnti, 2017). Finally, guidance on feedback was 
rooted in research suggesting that coaches often do not take an analytic stance with respect to 
their colleague’s instruction (Murray, Ma, & Mazur, 2009). 

To facilitate the uptake of these practices, we drew on research on organizational 
routines, which emphasized how routines structure the interdependent work of diverse 


practitioners (Stelitano, Russell, & Bray, 2017; Feldman & Pentland, 2003; Sherrer & Spillane, 


Learning from Adaptation 7 


2011). We embedded the key coaching practices in a designed routine — the Coach-Teacher 
Discussion Process (see Figure 2). The process begins with the coach and teacher selecting a 
high cognitive demand mathematics task and then familiarizing themselves with the task 
independently, including identifying the multiple ways that students would solve the task. The 
subsequent steps of the process consist of a pre-observation planning conference, a lesson 
observation, and a post-observation feedback conference. In order to gain competence in using 
the routine and to generate data for our research, we asked coaches to engage in the Coach- 
Teacher Discussion Process at least three times in year 1 and two times in year 2 of the project 
with two partner teachers (who had agreed to participate in our research). 

Finally, the coaching framework defines the inquiry stance as the way coaches should 
interact with the teacher. By specifying that coaches take an inquiry stance with teachers. we 
highlighted an interactional style that emphasized the coach relaying what they “noticed” about 
classroom practice and raising questions or “wonderings” instead of making more explicit 
recommendations for what teachers should do while teaching. The inquiry stance stems from 
research on professional learning which points to the need for active teacher participation in 
meaning making around shifts in practice (Opfer & Pedder, 2011). 

Coach development framework. Prior research and our practice-based partners 
experience both suggested that many instructional coaches get limited training in coaching 
practice (Gallucci, Van Lare, Yoon & Boatright, 2010). Consequently, a key design task was 
specifying a Coach Development Framework that included a curriculum and related tools for 
training coaches to enact the coaching framework. In the first two years of the project, coaches 
engaged in six face-to-face convenings each lasting two days. The research team presented 


emerging findings from analyses of data that coaches generated during action periods between 
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meetings and engaged coaches in conversations that aimed to make sense of data-based patterns. 
For example, our analysis of recorded pre-lesson planning conferences in an early cycle 
suggested only a few coaches were pressing teachers to articulate specific mathematics content 
that they wanted students to learn through the lesson they were planning; most coaches and 
teachers were simply identifying broad topics such as “equivalent fractions.” These analyses 
provided an opportunity for discussion of robust learning goals that help teachers prepare for 
lessons that support students developing conceptual understanding of mathematical ideas. Data 
analyses informed the design of learning sessions that dove deeper into the key coaching 
practices in the model drawing on artifacts from coaches in the network. Coaches also 
participated in a series of monthly Webinars between the face-to-face meetings. 

Continuous improvement: Learning about adaptation through inquiry cycles. A 
final key component of our coaching model is a commitment to continuous improvement. We 
refined the model through six improvement cycles, each launched by a face-to-face network 
meeting. During the improvement cycles, coaches generated and shared data that documented 
their coaching practice. These data included video-recorded coaching conversations and related 
artifacts such as instructional tasks and student work. The analysis of each cycle’s data 
contributed to refinement of the Coaching Model and to the training all coaches received in 
subsequent network meetings. Additionally, we trained coaches to utilize an inquiry routine — the 
PDSA cycle — to adaptively integrate the model into their local context. Both the research-based 
adaptations and PDSA cycles will be discussed further in the paper’s section on learning from 
adaptation. 

Evidence of efficacy of the model. As we strove for a scalable coaching model, a key 


consideration was generating evidence that the model had the promise of efficacy that would 
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make it worthy of scaling. In other words, we sought evidence that training coaches in the model 
contributed to improved coaching and improved teaching practice. In another paper, we 
examined our first two years of data that demonstrated improvements in coaching (ES=.71) as 
well as improvements in teaching (ES=.61) (Russell, et al., 2016). Longitudinal models for 
growth in coaching revealed no significant differences in growth rates among coaches — 
suggesting all coaches improved the depth and specificity of pre-conference discussions they had 
with their partner teachers at roughly the same rate. Meanwhile, longitudinal growth models for 
429 video observations from 103 partner teachers found an average effect size of .61 on a per- 
year basis for growth in teaching aligned with the coaching.” Furthermore, teachers with the 
lowest initial scores on our teaching measures grew the most. These findings provide promising 
evidence that the coaching model has potential to contribute to substantive practice 
improvement. A follow up quasi-experimental study, currently underway, is exploring whether 
there are significant differences in coaching, teaching, and student learning between coaches and 
teachers that were trained in our model and those that engaged in garden-variety coaching. 
Learning from Adaptation to Guide Effective Coaching at Scale 

In the prior sections we discussed how research on coaching informed the specification of 
our model for mathematics instructional coaching. In this section we turn our focus to the central 
topic of this paper: how our project approached studying and understanding the adaptation of the 
model as it was implemented by a network of coaches in diverse local contexts. 

Prior research about adaptation. While knowing more about effective interactions 
between coaches and teachers can contribute to improved coaching programs, improving 
education at scale is a more complex process than simply identifying optimal practices through 


research and directing practitioners to implement them (Coburn, 2003; Elmore, 1996). Policy 
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and implementation research has consistently found that when complex interventions are 
implemented, mutual adaptation occurs as educators change their work practices to align with 
the intervention and make adjustments to make the intervention useable in their context (Berman 
& McLaughlin, 1978; Coburn, 2004; Cohen, et al., 2008; Honig, 2006; Supovitz, 2008). 
Successful scaling requires that educators make reasoned adjustments to interventions as they 
become embedded into existing systems (LeMahieu, 2011; McDonald, Klein, & Riordan, 2009; 
Weinbaum & Supovitz, 2010). However, mutual adaptation tends to be the exception, not the 
norm, when educational reforms seek significant change in practice (Reiser et al., 2000). Local 
adaptation tends to be one-sided rather than mutual in that the intervention goes through more 
adjustments than do the individuals responsible for implementing it (Spillane, 1999). In a notable 
exception, Siskin (2016) draws on a study of the spread of the International Baccalaureate model 
to non-traditional school contexts — those serving low-income, students of color in an urban area 
— pointing to the potential for mutual adaptation in action, where feedback from researchers, 
context, and coaches enabled reformers to monitor implementation and adaptation, learn from it, 
and incorporate productive adaptations into designs (McDonald et al., 2009; Peurach, 2011). 

In the case of our coaching intervention, we expected two main types of adaptation: 
adaptation to diverse relational contexts and adaptation to diverse organizational contexts. Prior 
research on coaching pointed to potential relational challenges associated with coaching 
programs (Ippolito, 2010; Neufeld & Roper, 2003). Some teachers respond to coaching with 
resistance, perceiving coaching as evaluative or an indictment of their teaching practice (Bean & 
Carroll, 2006; Mangin & Stoelinga, 2011). As a result, coaches often avoid giving the critical, 
substantive feedback to teachers that challenges them to change their practice (Lord, Cress & 


Miller, 2003). Given these relational challenges, we reasoned that coaches would adapt their 
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work with teachers based on their impression of their openness to coaching. Prior research 
suggests that coaches vary in how they position themselves in relationship to teachers, 
suggesting two primary distinctions: responsive versus directive (Deussen, Coskie, Robinson, & 
Autio, 2007; Dozier, 2006; Ippolito, 2010). While different coaching models advocate different 
stances, few studies have investigated these claims, and when they have the studies have not 
directly investigated the efficacy of these stances (Ippolito, 2010). 

Prior coaching studies also point to features of organizational context that shape the 
implementation of coaching programs. For example, the selection and training of coaches 
influences the degree to which they are a source of teaching expertise for colleagues (Coburn & 
Russell, 2008). People who enter coaching roles may be relatively expert teachers, but they need 
learning opportunities to develop coaching skills and dispositions, such as how to facilitate 
conversations with teachers about their instructional practices and how to gather and analyze 
evidence to monitor changes in teachers’ instruction (Gallucci, et al., 2010; Neufeld & Roper, 
2003). Even when coaches are intentionally selected and trained, they may not be effectively 
utilized in schools. For example, Bean and colleagues (2010) found that principals often direct 
coaches to spend a significant proportion of their time engaged in managerial support activities, 
rather than protecting time for direct work with teachers. Additionally, principals play a critical 
role in promoting (or hindering) teacher engagement in coaching. In a large-scale study of 
literacy coaching, Matsumura and colleagues (2009) found that principals influenced coach 
access to teachers and classrooms through their level of endorsement of coaches as sources of 
expertise. Finally, school and district leaders shape the enactment of coaching roles in 
consequential ways by framing the purpose and goals of coaching programs (Mangin & 


Dunsmore, 2015). 
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Learning from Adaptation in the TN Math Coaching Project 

Our project aimed to learn from coach adaptation of the model in two ways. First, 
researcher-driven inquiry cycles attended to variation in implementation of the model and sought 
to identify adaptations that were associated with positive coaching and/or teaching outcomes, 
which could become part of the model’s design. McLaughlin (1987) noted that variability offers 
opportunities to learn about interventions and their implementation, noting that local responses 
generate natural experiments that should be exploited by analysts. In the course of conducting 
our traditional research in this project, we sought to exploit those natural experiments to 
contribute to our learning about the coaching model’s implementation. Second, practitioner- 
driven inquiry cycles provided a disciplined routine for coaches to engage in explicit adaptive 
integration. In this section, we describe our coach and teacher samples and then provide a broad 
overview of our approach to each form of learning from adaptation. In subsequent sections, we 
provide illustrative examples. 
Coach and Teacher Sample 

Through a competitive process including a written application and scenario-based 
interviews, we selected 32 coaches from 21 districts to participate in the network in year one of 
the study.* We selected coaches based on their possession of basic conditions that would enable 
engagement in the type of coaching our model promotes, considering factors such as adequate 
mathematics content knowledge and working in a role that afforded opportunities to engage in 
intensive one-on-one coaching. However, we strategically selected coaches to represent a range 
of initial coaching capacity and experience, and variation in basic contextual conditions such as 
geographic region, urbanicity, and role (e.g. school-based coaches versus district coaches). For 


all of the selected coaches, there was a gap between the type of coaching our model promotes 
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and their current practice, and for most that gap was quite significant. For example, our model 
emphasizes pre-lesson planning conferences, a practice many coaches did not employ. 

Coaches signed a contract with the TN Department of Education accepting the following 
responsibilities: attend three 2-day network meetings per year; participate in monthly Webinars; 
document all coaching activity using a Coaching Log; complete periodic short surveys; select 
two teachers for intensive study and commit to collect data about their practice; and commit to 
engage as partners in the continuous improvement of the Model. Each coach selected two partner 
teachers to participate in the study who taught mathematics in Grades 3 through 8, although one 
teacher left the study mid-year. Consequently, our sample consists of 63 partner teachers. 
Researcher-Driven Inquiry Cycles 

During each cycle, coaches implemented the coaching model in their work with two 
partner teachers, specifically they completed and documented one cycle of the Coach-Teacher 
Discussion Process with each partner teacher. As these data were shared with the research team, 
researchers analyzed key constructs related to the uptake of the model and the expected points of 
context variation including aspects of the relational context of coaching such as teacher 
responsiveness and the organizational context such as coach role construction (school-based 
versus district-based and number of teachers that coaches supported at any given time). 

Research-driven inquiry cycles culminated with evidence-based discussions with coaches 
and state leaders which enabled sense making about the findings. Subsequent training sessions 
provided ongoing guidance for implementation based on what we were learning from 
implementation. This process is consistent with the findings from the New American Schools 
implementation studies conducted by RAND which emphasized ongoing communication and 


support for implementation among designers and practitioners (Berends, 2000; Berends, Bodilly 
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& Kirby, 2002). Our discussion of Study 1 illustrates an example of researcher-driven inquiry 
into adaptation to relational context variation. 
Practitioner Driven Inquiry Cycles. 

Implementation that embraces mutual adaptation expects local variability and aims to 
give practitioners time and support to make good choices about how to implement interventions 
(McLaughlin, 1976). Our approach to helping coaches make good choices in their adaptation of 
the model drew inspiration from improvement science, which provides tools for learning how to 
improve practice at scale by building an evidence base generated from disciplined practitioner 
inquiry (Bryk et al., 2013). Improvement science has a long history in the manufacturing 
industries, and subsequently the healthcare fields, and provides a systematic methodology for 
learning from practice to improve the systems and processes that shape work within 
organizations (Berwick, 2008; Gawande, 2007; Langley et al., 2009; Lewis, 2015). 

Improvement science speaks to the need for adaptive integration of standard work 
process into new contexts, which is critical for learning how to scale improvements. As an 
innovation or change that has worked in one context moves into others, improvement methods 
are used to learn what it takes to make it work under diverse conditions (Bryk et al., 2015; 
Tichnor-Wagner et al., 2018). For example, schools participating in a network of districts aiming 
to reduce beginning teacher attrition — the Building a Teaching Effectiveness Network — used 
Plan, Do, Study, Act (PDSA) inquiry cycles to adapt a standard process for coordinating 
feedback for beginning teachers into the existing constraints of their school systems. The extent 
to which schools utilized improvement science methods to address the adaptive integration 
problem predicted their successful enactment of the feedback process (Hannan, et al., 2017). One 


way to plan for adaptive integration is to involve practitioners in designs for implementation 
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(Reiser, et al., 2000; Tichnor-Wagner et al., 2018). We issued invitations to our coaches to co- 
design the integration of the coach-teacher discussion process into their local contexts, through a 
process of coach-led inquiry cycles. While resource constraints precluded a comprehensive 
investigation of the uptake of these inquiry cycles, we documented strategically selected cases to 
begin to explore this process. Our presentation of exploratory Study 2 provides an illustrative 
example of coach-driven inquiry into adaptation to organizational context variation. 
Study 1: Researcher-Driven Inquiry into Adaptation to Relational Context Variation 
In this study, we demonstrate how we tracked the ways coaches adapted their practice 
when working with teachers that they perceived to have varying levels of responsiveness to 
coaching. In this way, it is a case of adaptation to the diverse relational contexts of coaching. 
Our exploratory analysis of year one data revealed evidence that coaches were adjusting their 
practice based on their judgments about the type of teacher they were working with: one judged 
to follow through versus one less likely to follow through. We found that when coaches rated 
teachers as relatively less responsive to coaching at the beginning of the year, those same 
teachers judged coaches to change the nature of their interactions during coaching over the 
course of the year. Specifically, coaches became more likely to provide more explicit directions 
about teaching. In contrast, when coaches rated teachers as relatively more responsive to 
coaching, the same teachers reported that coaches tended to increasingly press them to engage in 
mathematical reasoning over time. In other words, coaches’ perceptions of teacher 
responsiveness shaped whether they emphasized explicit direction about teacher actions or deep 
conversations about mathematics. 


Data Collection 
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For this study, we drew on three primary data sources: (1) video footage and artifacts 
from coaching cycles; (2) log entries completed by coaches for each interaction with their partner 
teachers; and (3) periodic surveys sent to partner teachers and coaches. Coaches and their two 
partner teachers participated in three formal coaching cycles in year one using the Coach- 
Teacher Discussion Process, described in the prior section. To capture intensive data on coaching 
practice, each enactment of the Discussion Process was documented by coaches who gathered: 
videotapes of pre-lesson planning conferences, lessons, and post-observation feedback 
conferences; teacher and coach planning and reflection notes; and artifacts such as the 
instructional tasks. Coaches shared these multiple data sources with the research team. 

To get a broader perspective on coach engagement with each partner teacher, coaches 
completed a web-based coaching log (Coach Tracker) every time they had an interaction with 
their partner teachers. These log entries consisted of a series of fixed response items related to 
the date, duration and type of interaction (e.g., informal conversation or demonstration lesson). 
The log also provided an opportunity for coaches to rate teachers’ responsiveness to coaching, 
specifically their judgment of the likelihood that teachers would follow through on the teaching 
ideas discussed during a given interaction. Coaches could also use a series of open-ended 
questions to note goals for teachers and reflect on their coaching practice. 

Finally, we administered the Discussion Process Survey to teachers following each time 
they participated in the Coach-Teacher Discussion Process. This survey included a short set of 
items that aimed to measure uptake of the key coaching practices in our model, such coaches 
having deep discussions of mathematics content, student learning, and pedagogy with teachers. 
Analytic Approach 


Our analyses for Study 1 explored whether coaches were adapting their practice in 
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response to their perceptions of teacher responsiveness. 

Measures. We explored items on the Discussion Process Survey in order to develop 
constructs that fit our conceptual model. One factor we call coaching explicitness included three 
items that if endorsed by teachers would suggest that coaches recommended a prescribed set of 
teaching practices when coaching teachers. For example, one item asked teachers the extent to 
which they agreed with the following statement: “As a result of this cycle, I know a set of 
prescribed behaviors that will always serve me well in the classroom.” A second item asked, 
“After this coaching cycle I could articulate the theory of mathematics teaching and learning my 
coach has.” Finally, a third item asked teachers the extent to which their coaching interaction 
was characterized by the following, “Most of the time was spent talking about the actions which 
I should engage in during the lesson.” Endorsement of these three items suggests that teachers’ 
perceived coaches as being explicit about specific teaching behaviors that should be enacted. 
Considering our coaching model, the explicitness construct is complicated to evaluate. On the 
one hand, the construct could potentially serve the goal of preserving integrity to the goal of 
increasing the depth and specificity of discussion of pedagogy (one aspect of the instructional 
triangle), given its focus on teacher actions. On the other hand, it runs counter to the goal of 
adopting an inquiry stance with teachers — where co-construction toward an understanding of 
how to improve teaching is ideally realized. 

A second construct we call coaching press, was composed of two items that measured 
teachers’ perceptions of whether they were “pressed” to discuss mathematics. These items asked 
the teachers the extent to which they agreed with the following statements: “I was pressed to talk 
about mathematical reasoning”, and “I was pressed to make sense of mathematical ideas”. There 


is no ambiguity in how these items are conceptually aligned to our coaching model’s aim to build 
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teachers’ capacity to think deeply about math content and students’ mathematical thinking. In 
this sense, we viewed the press construct as an indication that coaches were ensuring deep and 
specific conversations about the instructional triangle, while simultaneously engaging in co- 
construction of ideas during the inquiry process. 

Teacher responsiveness was assessed by coaches in each log of an interaction with a 
partner teacher on the Coach Tracker log. Specifically, an item asked coaches to rate teacher 
responsiveness using the following scale: (1) Teacher exhibited behaviors that signaled explicit 
resistance; (2) No explicit resistance but teacher was not prepared and/or signaled low 
engagement; (3) Teacher was compliant, but wasn’t open to suggestions and/or I don’t expect 
any follow through; (4) Teacher exhibited behaviors that signaled engagement, yet I am 
uncertain about their level of follow through; and (5) Teacher exceeded expectations for 
engagement and I have high confidence they will follow through. To generate scores for 
coaches’ judgments of teacher responsiveness for Cycle 1 we averaged across all coach logs 
before the end of Cycle 1. The distribution of coach reports of teacher responsiveness indicated a 
fair degree of variation — 15% of coaches reported an average at or below a score of 3, another 
10% were between 3 and 4, 34% indicated a responsiveness score of 4, 7% were between 4 and 
5, and the final 34% were at a score of 5. 

A final key construct in our analyses was a proxy for coaching quality and efficiency at 
Cycle 1. It was important to consider whether patterns in the relationship between perceptions of 
teacher responsiveness and coaching decisions about explicitness and/or press were influenced 
by the quality of the coaching interaction. Therefore, we constructed a pre-conference efficiency 
variable that utilized expert ratings of the pre-conference interactions between coach-teacher 


pairs (e.g., pre-conference rigor items included, deep and specific discussion of math content, 
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pedagogy and student thinking) divided by the length of the pre-conference. We then used these 
scores to predict both status and linear change on explicitness and press over the course of the 
year. Efficiency was included as a covariate as a competing explanatory theory of whether and 
how coaches adjusted their relational interactions with teachers. Thus, we added this covariate 
thinking that coaches are weighing a lot of factors when making decisions about how to adapt 
their coaching. One mitigating factor could be coaches’ perceptions of the need for rigorous pre- 
conference conversations as well as the need for brevity to use time efficiently. Similar to 
classroom teaching, we hypothesized that the greater the perceived need for efficiency, the more 
likely it was that the coach would exhibit more explicitness in coaching conversations. 
Exploratory survey analyses. We engaged in exploratory analyses to examine whether 
coaches adapted their interactions with teachers in response to the variation in the relational 
context of coaching measured by coach’s perceptions of teacher responsiveness. We looked for 
evidence of adaptation by examining whether teachers reported changes in the coach-teacher 
interactions. We employed multi-level growth models for both the coaching explicitness and 
coaching press scales as outcomes, nested in three time points over the year, and time points 
nested in coach-teacher pairs (for a full description of the model, see Appendix A).* We 
hypothesized that if coaches adapted their coaching to individual teachers it might be based on 
their judgments about how responsive the teacher appeared to be in their initial interactions (i.e., 
whether they believed the teacher would follow-through with the ideas they discussed together). 
Specifically, in an initial model, we used the measure of coaches’ initial perceptions of teacher 
responsiveness to predict changes in teachers’ perceptions of the coaching they received 
(explicitness and press for math reasoning). We ran a second model, adding another independent 


covariate measuring the efficiency of the first pre-conference. We also included several 
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covariates as controls in these models. First, we included each of the items to adjust for any 
differences in item difficulty within each scale. Next, we included the length of the task, or the 
amount of time it took for the teacher to implement a given task as measured through videotapes 
of classroom instruction, as a time-varying covariate.> We assumed that this was a proxy for the 
nature of the task being implemented. We included this because we further assumed the nature of 
the task itself could influence partner-teacher perceptions of the focus on explicitness or press, 
with higher cognitive demand tasks (as evidenced by longer time spent on the task) providing 
more fruitful opportunities for coaches to press teachers for mathematical reasoning. 

Qualitative case-based analyses. To explore whether the statistical phenomenon was 
evident in interactions between coaches and teachers, we examined a limited number of 
qualitative cases, analyzing transcripts of coach-teacher interactions in pre-conferences. Cases 
were selected that most represented the pattern in our survey data. Three explicitness cases 
included coach-teacher pairs where the coach rated the teacher as relatively less responsive in the 
first cycle, and then teachers subsequently reported coaches provided explicit teaching direction 
in later cycles. The two press cases included teachers rated highly responsive by their coaches 
and where the teacher reported coaches increasingly pressed them to engage in math reasoning. 
We coded evidence of responsiveness in the first cycle pre-conferences and then coded for 
evidence of explicitness and press for mathematical reasoning in the second and third cycles. We 
present examples from one case representing each pattern. These qualitative cases should be 
interpreted cautiously, not as evidence that this pattern of adaptation was present in all coach- 
teacher interactions, but rather that we could see what these phenomena looked like in a select 


set of cases. 
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Survey results. We first explored psychometric properties of our primary measures in a 


fully unconditional measurement model. Results indicated that teacher reports of explicitness (A1 
= .752) and press (A2 = .818) form reliable estimates of between-teacher differences in status® — 


which is a relative measure of variance between teachers at a given time point compared with the 
measurement error (67). These HLM reliabilities demonstrate that there is variance in how 
teachers answered items on each scale, and further, that the variance is relatively large when 
compared with the variance between items on each scale within teachers. Interestingly, 
correlations between slope estimates for explicitness and press indicate a negative correlation (r 
= -.60), suggesting the two scales are somewhat independent, but that for at least some teachers 
while one scale was increasing, the other scale was decreasing. 

Exploratory findings on the press slope: The top half of Table 1 provides evidence that 
coaches adapted their coaching based on their early judgments about their partner teachers. Take 
for example, the findings for teacher responsiveness on the construct press in both models | and 
2. Here, we find that when coaches deemed teachers to be more responsive at Cycle 1, the 
partner teachers they coached reported a higher slope for changes in press (111 = .48, s.e. = .26, 
p = .074 in model 1; Bi11 = .48, s.e. = .27, p = .079 in model 2). Thus, when coaches deemed 
teachers to be more responsive initially, the nature of their interactions with teachers changed 
over the course of the year. Teachers reported greater amounts of press over time on the 
discussion process survey. In model 2, the variable measuring efficiency of the pre-conference at 


Cycle 1 is not significant on the press slope. 


Insert Table 1 


Exploratory findings on the explicitness slope: Findings on the explicitness slope provide 


a nice complement to those on the press slope. The less likely coaches deemed teachers to 
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follow-through with ideas discussed in their initial interactions, the higher the slope for changes 
in coach explicitness over the year (B211 = -.53, s.e. = .20, p = .011 in model 1; B11 = -.53, s.e. = 
.19, p = .008 in model 2).’ Once again, we see evidence of adaptation based on coach judgments 
of teacher responsiveness. The pattern of findings for the coach explicitness and press scales 
suggests that coaches are adapting their coaching practice in response to their judgments of 
teacher receptivity to coaching when they begin working with an individual teacher. 

Finally, the variable measuring the efficiency of the pre-conference at Cycle | had a 
similar effect and interpretation, only in the opposite direction (B212 = 1.14, s.e. = .48, p = .019). 
The more efficient the initial pre-conference was, the more the partner-teacher reported the 
coaching interactions to increase in their explicitness over the course of the year. It is interesting 
that both lower teacher responsiveness and the higher measure of efficiency predict higher rates 
of explicitness in coaching over the year. It suggests that each covariate predicts some unique 
variance in the slope, and underscores the point that coaches likely attend to multiple factors as 
they make decisions about how to adapt their interactions with teachers in their efforts to help 
them improve their instruction. 

Qualitative results. Our qualitative analyses revealed evidence of the survey-based 
findings. While we did not systematically analyze the full transcript dataset, we present evidence 
in these strategically sampled cases to show what this type of adaptation looks like in practice. In 
a case that we believe represents our explicitness findings, a coach rated her third-grade partner 
teacher relatively low on responsiveness during the first cycle: the coach’s evaluation of teacher 
responsiveness for each interaction in the first cycle averaged 3.33 or the equivalent of a 3 on 
rating scale signaling, “teacher was compliant, but wasn’t open to suggestions and/or I don’t 


expect follow through.” We saw some evidence of this teacher’s possible compliance orientation 
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to coaching in the first cycle pre-conference: 37% of the teacher’s turns in talk were one-word 
responses such as “Mhhm” or “OK”. 

Perhaps adapting to her perception that the teacher would not follow through, the coach’s 
moves in the second and third cycle are much more explicit than in the first cycle. Throughout 
the second and third conference, the coach tightly controls the flow of the conversation and 
makes specific recommendations about what the teacher should do when teaching the upcoming 
lesson. For example, the coach presses the teacher to articulate a pedagogical goal for the lesson 
by saying, “Alright, so what about you, personally, what are your goals as a teacher for this 
lesson?” The teacher responds by talking about the students, saying “Well my goal is for them to 
be able to obviously answer all four questions, but show me with the provided manipulatives 
how much of each color yarn that you need.” The coach then redirects the teacher to think about 
goals for her teaching: “Alright, but, for your teaching practice?” The teacher replied by 
articulating a goal: “OK, for me, I’ve looked over this and I think that what I need to work on, is 
a lot of times I give struggling students not enough think time, so I need to support productive 
struggle in learning math.” The coach accepts this goal and discusses it with the teacher, but then 
suggests another goal for the teacher: “And another part of that, may be an action plan for you 
would be to be purposeful in planning your assessing and advancing questions.” 

Additionally, the coach makes several direct suggestions phrased in a way that suggests 
that the coach may have been trying to hold the teacher accountable for following through on 
these pedagogical goals. For example, after a discussion of the second pedagogical goal in the 
conference referenced above, the coach recaps by saying, “Alright, so advancing questions and 
then walking away are going to be your action steps tomorrow to let them struggle.” This 


discussion continues to the third cycle conference when the coach said: 
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In the share phase, and we talked about this during your last coaching cycle, we 

talked about how you wanted to take more of a backseat and let the kids lead the 

share phase. So, as we went through all those solution paths, what do you think 

about kind of having a checklist, like I wrote down here all the solution paths, and 

be intentional about looking for those so when you get to the share phase, you 

know who you’re calling on, and you’re thinking about, “Ok, I want to pick a kid 

who showed the arrays. I want to pick a kid who showed the area model. I want 

to pick a kid who simply wrote the equations. 

As these passages illustrate, the interactions in this conference reflect a focus on the what the 
teacher should do, prompted by explicit coach suggestions. 

In contrast, our second focal case illustrates a coach working with a third grade teacher 
that she judged as highly responsive to coaching, receiving a rating of 5 on all interactions in the 
first cycle which correspond with, “teacher exceeded expectations for engagement and I have 
high confidence they will follow through”. In this case, we see a very different pattern of 
interaction, one with fewer coach moves that suggest explicit actions the teacher should take in 
the classroom and more focus on a general press for teacher and student mathematical reasoning. 
In the first cycle pre-conference discussion, the positive rapport between coach and teacher was 
evident in their ease in communication, which included a strong coach orientation toward 
listening to what the teacher says, and a general upbeat and positive interactional style. The 
coach repeatedly presses the teacher to articulate her thinking about mathematics and reason 
about students’ mathematical thinking. For example, when the coach first asks the teacher about 


her mathematical goals for students, the teacher says: 
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I believe the last time we had done this we had at least one group that really were 

able to show that they had a conceptual understanding of what we were doing 

with decimals...I expect to see good representation of fraction models. I expect 

to see some — and hear some really great explanations of thinking. And then also 

—and this is going back to the evaluating others — some really good constructive 

peer feedback. 
In this conference, the teacher is making general statements about the mathematical goals of the 
lesson. However, the coach does not accept these general statements, and goes on to press the 
teacher throughout the conference to get deeper and more specific. For example, in this passage 
the coach and teacher co-construct a deeper statement about the mathematical goals: 

Coach: It really wrestles with interpreting a fraction as division of the numerator 

by the denominator because students so many times don't see that. They see 

three-fourths — as fractional form. 

Teacher: Yes. 

Coach: And don't understand that represents — 

Teacher: We are struggling with getting them to conceptualize doing — how to 

actually divide a line into equal sections. That is — honestly, that is a very 

important skill that they struggle the most on. They can read one with all the lines 

already there, but getting them to conceptualize how to split it up is very difficult. 

Coach: And — yeah, it's — they don't understand that three-fourths means three — 

Teacher: Over four. 


Coach: — divided by four. 
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At the end of a conference where the teacher has been repeatedly pressed to articulate the 
mathematics goals of the lesson and how students will make sense of the mathematics, we later 
see the teacher taking the lead in this kind of talk. 

Teacher: So I think in a lot of ways this is really going to tell us who is still at the 

elementary level in terms of who do we really need to go back and work on the 

conceptual understanding of partial amounts. And then I think have a clear 
distinction as to who has it, who doesn't, and where it is that they're missing... 

Coach: So the first thing, what's the big mathematical idea here that they have to— 

Teacher: They have to compare values. 

Coach: But to do that, what do they have to do? 

Teacher: They're gonna have to come up with like denominators. 

Coach: Yeah. Equivalent fractions. 

While we cannot be certain about the coach’s intentions, we speculate that the coach’s 
perception that the teacher was responsive to coaching, may have contributed to her 
focus on pressing the teacher to focus on mathematical reasoning, rather than specific 
pedagogical moves the teacher should make in the classroom. 

In sum, these two cases provide some indication of what these survey-based 
adaptation patterns look like in the interaction between coaches and teachers. In both 
cases, the coach plays an active role in shaping the conference. In the first case, 
however, the coach is more directive in terms of what behaviors she expects to see the 
teacher do during the lesson while, in the second case, the coach exerts her influence in 
terms of urging the teacher to uncover mathematical ideas that are undergirding the 


lesson itself and student thinking about those ideas. 
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Relationship to teaching improvement. We found it interesting that both the 
explicitness and press constructs showed evidence of coach adaptation. However, while 
adaptation toward press was viewed theoretically as unambiguously positive, adaptations toward 
explicitness were more indefinite. Thus, while we were heartened to find that some coaches were 
increasingly pressing teachers for mathematical reasoning, we were less certain about 
adaptations toward explicitness because it could be an indication that some coaches were not 
implementing the Model’s inquiry stance with integrity. As a result, we examined how teachers’ 
perceptions of coach explicitness was related to teaching improvement trajectories. 

For this analysis we first examined the teachers’ reports on the discussion process survey 
and categorized them based on their pattern of responses. Nineteen teachers were categorized as 
reporting increasingly greater explicitness in interactions with their coach over the course of the 
year. We then examined whether teachers who reported increasing explicitness had different 
patterns of teaching improvement — on our video-based measure of students’ opportunities to 
engage in conceptual thinking - compared with the remaining teachers. We found no differences 
in growth patterns for these 19 teachers relative to all other teachers.® This suggests that, under 
the right conditions, explicitness may be efficacious; as such, it called for reconsideration (or 
elaboration) of the principle of taking an inquiry stance with teachers, a component of the 
coaching model’s design. 

Study 2: Coach-Driven Inquiry into Adaptation to Organizational Context 

In the first year of the study, we focused primarily on training coaches to take up our 
coaching model and iteratively refining our perspective on the key coaching practices based on 
the data that coaches submitted capturing their work with their partner teachers. As the coaching 


framework coalesced around three key coaching practices and a central coaching routine, the 
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work in our second year actively engaged coaches in the adaptive integration of the coaching 
framework in their diverse local contexts. In this exploratory study, we reviewed documentation, 
albeit limited, to explore strategic cases of robust coach use of disciplined inquiry to support 
model adaptation. 

Data Collection 

In preparation for our adaptive integration work, we collected several data sources to 
learn about the variations in local contextual conditions that might influence implementation of 
the coaching model. At the beginning of year one, we had coaches write essays describing their 
coaching role and context. In an end of year one Context Survey, we asked coaches and their 
partner teachers to reflect on the way that school and district context shaped their use of the 
coaching model. We triangulated survey-based accounts with interviews of partner teachers also 
at the end of year one; these interviews focused on their experiences working with a coach that 
was implementing our model. In addition, we gathered information about an important 
consideration for adaptive integration of the model-the way that schools and districts defined 
coaching roles—through our ongoing interactions with coaches throughout the year. 

Finally, we documented coaches work in PDSA inquiry groups where they experimented 
with adaptive integration. We encouraged coaches to document their engagement in PDSA 
cycles by completing a PDSA Tracking Form. A limitation of our study was the incomplete 
documentation and submission of these forms. In general, project demands on coaches were 
high: they documented coaching cycles by gathering and uploading video recordings and related 
artifacts such as assignments and student work, and completed the coach tracker log, and several 
surveys throughout the year. The addition of the PDSA tracking form came on top of these other 


documentation expectations. We documented collaborative work in inquiry groups through 
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observation and artifacts. These data suggested most teachers were engaging in PDSA cycles, 
despite incomplete documentation and submission of forms, but the observational data did not 
provide us with consistently detailed of information about their PDSA work. As a result, we 
focused on selecting cases where we had the most complete data. We do not claim these cases 
are representative of coaches work in the project, but rather cases of the productive use of the 
PDSA inquiry routine. 
Analytic Approach 

We laid the groundwork for our adaptive integration work by analyzing the school and 
district factors that shaped coaches work. First, we categorized coaches with salient role-based 
descriptors based on what they wrote in their context essays including, school-based or district 
based, number of schools they worked with, and the defined focus of their work (e.g., teacher 
support versus student intervention). We conducted descriptive analyses of the Context Survey 
administered to coaches and teachers and thematic analyses of interviews with partner teachers 
were undertaken to identify other salient dimensions of context variation. Then, we generated 
case studies of individual coach’s engagement in adaptive integration through within coach 
analysis of data and follow up conversations with these coaches. 
Results 

We begin by describing our findings related to variation in coaching roles and the most 
influential school and district conditions reported by coaches. Our coaches varied considerably in 
the way their role as instructional coaches were defined and operationalized. In year one of the 
project, most coaches (50%) were school-based mathematics coaches working with one or more 
schools. However, the remaining half of coaches were not only responsible for mathematics 


coaching, but also had broader job descriptions such as school-based instructional coaches 
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responsible for coaching in both mathematics and literacy or district-based coaches working with 
a number of schools and sometimes multiple content areas. While 13 out of 32 coaches worked 
with a single school, the remaining 19 worked with teachers in two or more schools, with nearly 
half of the 19 working with three or more schools. Two-thirds of our coaches worked exclusively 
with elementary grades mathematics teachers, but the remaining third (N=13) worked with 
teachers in Grades K-12 and or middle and high school teachers resulting in greater task 
complexity due to more complex or a wider range of instructional content. The fact that coaches 
had duties in more than one content area, working with a wide range of disciplinary content 
and/or multiple organizations contributes to the complexity of the type of coaching work we are 
promoting and in so doing places demands on coaches to find ways to integrate the coaching 
model’s key practices and routines into already complex job responsibilities. 

Overall, coaches and partner teachers tended to report alignment with the coaching model 
and the priorities expressed by school and district leaders. That said, one-third of coaches 
reported lack of agreement between principals’ visions of effective mathematics instruction and 
the vision promoted by our coaching model. This lack of alignment was also voiced by a sizeable 
minority of our partner teachers through their responses to surveys and interviews. For example, 
one teacher said, “The goals were different. My principal’s goal is we need to make our growth 
[on state achievement tests], and my coach’s goal for me was to deepen my understanding in 
mathematics content and work on teaching in that Common Core style.” The most salient 
contextual challenge that emerged from both teachers and coaches was the beliefs and resistance 
of other teachers in their schools. While coaches generally reported that their partner teachers 
supported the style of teaching promoted in the coaching framework, coaches and teachers 


reported that other teachers in their schools tended to emphasize “more procedural” or 
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“traditional” teaching. This suggests an important consideration for the scale up of our coaching 
model, as coaches will need to work with teachers with a wider range of buy in and beliefs. 

Responding to our emerging understanding of the contextual challenges faced by our 
coaches, we actively engaged our coaches in experimentation aimed at promoting the network’s 
learning about how the model can be adaptively integrated into diverse local contexts. We 
trained coaches to use the Plan, Do, Study, Act inquiry routine to guide their experimentation 
with ways to adapt their use of the model while preserving its integrity to the key coaching 
practices.? Coaches volunteered to join groups organized around contextual challenges, that were 
derived from their compiled survey responses which we shared with coaches in a face-to-face 
network meeting. We initially presented coaches with a list of contextual challenges we 
generated from survey analysis and coaches decided which implementation challenges they 
wanted to work on. See Table 2 for the list of groups that formed and the number of coaches that 
selected to work in each group. In their inquiry groups, coaches collaboratively planned inquiry 
cycles and shared what they learned from them in subsequent meetings. 

In the next section, we provide one illustrative case of PDSA work undertaken by a coach 
working in the negotiating time challenges group. When given a choice of an implementation 
challenge to address in work with a collaborative group, 80% of coaches at the fourth network 
meeting indicated negotiating time challenges was one of their top three problems of coaching 
practice, and 23% said this was their number one implementation challenge. We highlight this 
case for several reasons. It illustrates a coach working on a salient implementation challenge for 
the network, and several coaches reported trying some of the strategies our focal coach designed 
and tested. Additionally, this coach was relatively more complete in her documentation of her 


iterative PDSA cycles giving us a rich portrait of the way coaches might use disciplined inquiry 
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to support uptake of the model with integrity. We emphasize that this case represents exceptional 
engagement in the PDSA process in our network, rather than being representative of the work 
that all coaches did. However, exceptional cases can generate important learning particularly for 
informing our understanding of emergent phenomenon such as the use of improvement science 
in education. 

Managing time to enable intensive coaching interactions. The coaches in our network 
reported complex and diffuse role responsibilities. In essays about their coaching role and local 
context that we asked coaches to write at the beginning of the project, they described engaging in 
a wide array of tasks including direct support for teacher development through observing and 
providing feedback, co-teaching, and orchestrating professional development sessions, 
intervention support for students, curriculum and assessment development, data analysis and 
management, as well as general administrative tasks. Our model prioritizes coach engagement in 
intensive one-on-one coaching cycles, which require approximately 2 or 3 hours of direct 
interaction with each teacher per cycle, plus approximately 1 hour of additional preparation time. 
Consequently, we heard from some coaches that they struggled to prioritize time for engagement 
in the Coach-Teacher Discussion Process and ended up having to frequently cancel and 
reschedule conferences or observations during cycles. 

For example, Martha was one of the coaches in our network who identified time 
constraints as her biggest challenge to implementing the model. As the only mathematics coach 
in her small, rural school district, Martha was charged with working with all mathematics 
teachers in the three schools in her district. Martha’s role included supporting teaching 
improvement through modeling, co-teaching, providing feedback after observations, and leading 


professional learning community meetings. But she also provided direct instruction during 
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intervention time for struggling students in Grades 4, 6 and 8, developed common assessments 
and analyzed student data, and provided PD on the district’s math curriculum. In fact, she 
reported that she routinely spent 50% or more of her time on administrative tasks rather than 
direct support for teachers and students. Martha received some direction from school and district 
leaders to focus on providing support to novice teachers, teachers with the lowest evaluation 
scores, and teachers in Grades 2 through 8. Like most of the coaches in our network, Martha’s 
job was complex and few structures were defined to support her management of this role 
complexity. 

As Martha began to utilize the Coach-Teacher Discussion Process to structure her one- 
on-one support for teachers, she found that she was having trouble reliably enacting the routine. 
For example, she completed a pre-observation lesson planning conferences with a teacher, but 
was not able to observe the teacher teaching the associated lesson, because her principal pulled 
her into a meeting with another teacher without advanced notice. Having missed the teacher 
teaching the lesson, the coaching cycle had to be restarted, because it is anchored in planning, 
teaching, and reflecting on a specific mathematical task. Another time, Martha had to cut short 
her time allotted for a post-observation feedback conversation, which did not enable her to have 
the deep and specific conversation she had hoped with the teacher. 

Facing this challenge, Martha was eager to join the time constraints PDSA work group to 
learn how she could protect time for engagement in the Coach-Teacher Discussion Process. 
Through a series of linked PDSA cycles, Martha identified, tested and refined an adaptation to 
the coaching model. She first tested a weekly calendar (posted in a Google Doc) that specified 
her coaching schedule for the week, which she shared with school and district administrators. 


While administrators acknowledged receiving the calendar, in a subsequent week she missed six 
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scheduled coaching events. Five missed events resulted from administrators directing her to 
other tasks (e.g., preparing assessment copies), despite having access to her calendar. 

Consequently, Martha added a step to the scheduling routine that included a brief weekly 
report sent via email to school and district administrators at the end of each week, summarizing 
interruptions to her coaching schedule. Martha theorized that seeing a connection between the 
competing demands made for her time and missed opportunities for coaching would help 
administrators think more carefully about whether to interfere with scheduled coaching events. 
As predicted, administrators began consulting her web-based coaching schedule and 
administrators’ requests to miss coaching events diminished over time. 

The schedule routine became a key component of Martha’s coaching practice, helping 
her to protect scheduled coaching time from interruption from school and district administrators, 
coordinate her multiple responsibilities, and take up the coaching model with integrity. Martha 
found that systematically tracking and sharing the number of schedule disruptions helped her 
iteratively refine the scheduling routine, and know when she arrived at an adaptation that 
improved her uptake of the model with integrity. 

Additionally, there is some evidence that Martha improved her consistency in both 
quantity and quality of coaching she provided. For example, her pre-conferences in year 1 ranged 
from 10 to 26 minutes long, with an average of 19.65, suggesting she was struggling to preserve 
time for deep and specific conversations. However, once she began PDSA work in year 2, her 
pre-conferences were all 24 minutes or longer, with an average of 26 minutes. Perhaps reflecting 
her struggle to protect time for coaching, the depth and specificity of her pre-conferences in year 
1 was inconsistent with two conferences scoring 7-points out of 9 on our rigor rubric, but others 


scoring in the 2- to 4-point range. This suggests Martha had the skill to conduct pre-conferences 
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with deep and specific conversations, but could not consistently enact this practice, perhaps due 
to implementation constraints. However, by the end of year 2, she was consistently engaging 
teachers in deep and specific conversations in pre-conferences. 

Martha shared her successes with the routine in a network meeting and subsequently 
other coaches incorporated a similar routine into their practice. In this way, calendaring routines 
became an adaptation to the model, or more aptly an elaboration, that addressed the challenge of 
integrating intensive one-on-one coaching into diverse local contexts. 

In conclusion, we have evidence that PDSA cycles can help coaches learn how to enact 
the model with integrity in schools with varying contextual conditions. While we cannot make 
claims about the pervasiveness of the productive use of coach driven PDSAs for model 
adaptation in our project through this exploratory study, exceptional cases such as Martha’s 
suggest that this is a potential way to support adaptation and the network’s capacity to learn how 
to scale up the coaching model. 

Discussion and Implications 

Education studies going back to at least the 1970s acknowledge the importance of 
adaptation during implementation emphasizing the need for both designs and local practice to 
change in ways that are in keeping with the core intent of the policy or innovation, but few 
studies find instances of this kind of mutual adaptation in practice (for an exception see Siskin, 
2016). Instructional coaching is a promising intervention to support mathematics instructional 
improvement at scale, but prior research on coaching implementation points to variable uptake 
due in part to contextual variation that presents implementation challenges. This history suggests 
that effective expansion of coaching programs likely requires that local educators change their 


practice in tandem with the adaptation of designed coaching programs to varying context 
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conditions. Our research and design process exemplifies a dynamic approach to the 
implementation and scaling of a coaching model which can support mutual adaptation. 
Contributions to understanding the role of adaptation in scaling up coaching programs 

Our findings point to key features of a scalable coaching model. First, coaching models 
must identify the key coaching practices that constitute the essence of the model. Clarity about 
key coaching practices provides a focus for supporting changes in coaching practice that the 
model aims to promote. But identifying key coaching practices should be a dynamic process 
unfolding as changes in coaching practice are monitored during implementation and scaling, and 
design conjectures about the relationship between key coaching practices and valued outcomes 
are tested empirically. Implementation research should attend to adaptations coaches make to the 
core practices when they confront implementation challenges, and should systematically test the 
robustness of practice adaptations by assessing whether they contribute to or inhibit valued 
outcomes (e.g., teacher learning and improvement). Through iterative cycles that test the 
adaptations coaches make in practice, we can learn what truly constitutes the essence of the 
model. 

In our project, we organized iterative research and design cycles that enabled us to test 
whether our design conjectures about the key coaching practices were associated with 
improvements in teaching. We were also able to identify ways that coaches adapted the core 
practices during implementation. This work was enabled by a comprehensive measurement 
system that systematically documented coaching and teaching practice, and rapid analytic cycles 
during our design phase. Rather than assume that coaches’ adaptations of the key coaching 
practices were lethal mutations, we sought to interrogate our assumptions about what practices 


contributed to the teaching changes we sought to support. 
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Our findings from Study lillustrate one form of adaptation coaches employed when 
implementing our model. Coaches adapted their use of the key coaching practice that promoted 
taking an inquiry stance during coaching conversations, utilizing questioning to promote teacher 
practice change, rather than more directive statements about what teachers should do when 
teaching. We found that coaches adapted their use of the inquiry stance in response to 
perceptions of teachers’ responsiveness to coaching. When coaches judged teachers to be more 
responsive initially, teachers reported that coaches increasingly pressed them to reason about 
mathematics and student learning during pre-lesson planning conferences (i.e., coaches were 
observed adopting an inquiry stance). However, when coaches judged teachers to be less 
responsive, adaption toward greater explicitness was observed. 

In our exploratory analyses of teaching, we show that coaches’ adaptation toward 
explicitness does not appear to be a lethal mutation, as growth among those teachers was 
consistent with all other teachers in the sample. Analyses, such as this example, that explore 
coaching adaptations as they relate to teaching improvement help us to think about which 
adaptations preserve (or violate) the integrity of the coaching model. Through iterative 
examination of the uptake and adaptation of the model’s key coaching practice we gained 
insights into the threshold of adaptation where the integrity of the model might be lost. In this 
case, we see evidence that deep and specific conversations about the instructional triangle is the 
key coaching practice to preserve in the spread of the coaching model, providing guidance for 
efforts to spread and scale mathematics coaching programs. 

The finding that coaches adapted their coaching to perceptions of teacher responsiveness 
also contributes to our understanding of how coaches adapt their work to accommodate different 


types of teachers, or what we have called the relational contexts of coaching, perhaps pointing to 
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an implicit developmental theory of stages of coaching. For example, depending on judgments of 
where teachers are in their thinking and their capacity to follow-through, coaches might provide 
teachers with more clear and specific directions about how they can develop their teaching 
practice in the short-term. These interactions could remain fairly brief and could be precursors 
for further learning about how to teach more conceptually in the future. On the other hand, 
teachers deemed highly responsive might be ready to engage in deeper learning discussions, 
where coaches press teachers for their mathematical reasoning, with the hope that by doing so, 
teachers will engage in similar practices with their students. This finding adds nuance to a 
literature on the relational contexts of teaching that has tended to advocate either a directive or 
responsive stance to coaching (Costa & Garmston, 2002; Dozier, 2006; Deussen et al., 2007). 
We should interpret our exploratory findings with caution given they are correlational not causal, 
and primarily based on teachers perceptions of coaching, but they have value in providing a 
novel relationship that can be explored in future studies. As we continue to explore the efficacy 
of more or less explicit forms of coaching, we contribute to the field’s understanding of the 
microprocesses of effective coaching practice (Gibbons & Cobb, 2016; Huguet. Marsh, & 
Farrell, 2014; Mudzimiri, Burroughs, Luebeck, Sutton, & Yopp, 2014) and much needed theories 
of mathematics teaching development (Munter & Correnti, 2017; Adler, Ball, Krainer, Lin, & 
Novatna, 2005; Gellert, Espinoza, & Barbé, 2013; Goldsmith & Schifter, 1997). 

Like our study of variation in teacher responsiveness, Study 2 explores another common 
coaching implementation challenge: managing varying and competing professional 
responsibilities. Martha’s experience is consistent with what we know about the complexity of 
the coaching role stemming in part from the need to find ways to efficiently support multiple 


teachers while managing time constraints and multiple demands (Galey, 2016; Mangin & 
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Dunsmore, 2015; Mudzimiri et al., 2014). Many coaches need to continuously manage their 
relationship with administrators to carve out the maximum amount of time they have to work 
with teachers, because they are asked to fill other administrative, non-coaching duties (Bean et 
al., 2010; Camburn, Kimball, & Lowenhaupt, 2008; Galluci et al., 2010; Mangin, 2009). Even 
when they work in supportive environments, coaches are faced with issues of efficiency. How 
many teachers can they reasonably expect to work with? How much time does it take on average 
to maintain productive contact with teachers? Do some teachers need more or less intense 
contact in order to make measurable improvement? Balanced with issues of efficiency are issues 
of effectiveness. What are the most effective progressions for taking teachers of a particular type 
through a learning progression? Does that learning progression vary for teachers of the same 
type? Coaches likely confront these questions, and more, subconsciously every time they engage 
with teachers. 

Martha’s work illustrates how coaches might use methods of disciplined inquiry to learn 
how to manage difficult implementation conditions (Tichnor-Wagner et al., 2018; Cohen-Vogel, 
et al., 2015). By making small and incremental changes to the coaching system in her district, 
Martha provides insight into ways coaches can proactively engage in the spread of a coaching 
program. By making barriers to her deep engagement with teachers visible to administrators 
whose actions were interfering with her time dedicated to coaching teachers, Martha’s inquiry 
cycles enabled her to engineer better contextual supports for rigorous coaching practice. While 
our case study approach provides some insight into the adaptive integration process, a key 
limitation is that we do not systematically explore the efficacy of the adaptions that coaches 
generated in PDSA groups. Further research that explores how adaptive integration can 


contribute to improvement outcomes is warranted. 
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Contributions to our understanding of reform adaptation more generally: An evidence- 
based approach to promoting mutual adaptation 

Based on our work in the TN Math Coaching Project, we posit a potentially generalizable 
process for evidence-based approaches to promote mutual adaptation, in the context of research- 
practice partnerships. Our experience suggests the potential power of beginning a collaborative 
reform effort with continuous improvement research aimed at identifying the core features of a 
reform that contribute to valued outcomes. In the TN Math Coaching Project, we closely 
monitored and documented implementation of our coaching model and then studied how 
variation in implementation related to coaching and teaching improvement. These iterative 
investigations facilitated identification of the essential core of our reform: deep and specific 
coaching conversations about pedagogy, content, and student thinking. Subsequently, we 
monitored implementation by tracking how coaches maintained the integrity of the model 
through enactment of this key coaching practice. 

Once the essential core of a reform or interventions is better understood, our experience 
suggests it is then important to build educator capacity for productive adaptation. In our project, 
we defined productive adaptation as those that could be enacted while still maintaining integrity 
to the key coaching practice, and we continued to train and support coaches in enactment of this 
practice so that they developed a deep understanding of the core features of the model. In 
tandem, we trained coaches in a principled way to experiment with supportive elaborations of the 
model that enabled them to head off potential implementation barriers that could result in lethal 
mutations. Martha’s case suggest that practitioners can learn how to implement research-based 


interventions under varying contextual conditions, through iterative inquiry cycles. More 
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comprehensive studies of adaptive integration would contribute to our understanding of the 
adaptive integration process. 

This process has several implications for promoting mutual adaptation in research- 
grounded reform efforts. The process model suggests that researcher-driven and practitioner- 
driven learning about adaptation can work in tandem to produce designs that are well positioned 
to be taken to scale. Mutual adaptation is one way to conceptualize scale, by focusing on 
variation in implementation and its contribution to local effectiveness (Coburn, Catterson, Higgs, 
Mertz, & Morel, 2013). Researcher-driven investigation of educators’ adaptations can identify 
what constitutes the critical elements of an intervention. Understanding the essential core of a 
reform or intervention clarifies what educators should be trained to do and what should be 
monitored during implementation (Kisa & Correnti, 2015). To do so requires a comprehensive, 
practical measurement infrastructure to support rapid analytics, and also suggests the value of 
iterative research and development cycles (Bryk et al., 2011; Tichnor-Wagner et al, 2018; 
Yeager et al., 2011). 

Practitioner-driven learning from adaptation can generate potential elaborations to an 
intervention that enable its implementation with integrity in varying contexts. This points to the 
need to build educator capacity to systematically identify and test adaptations. Accordingly, 
adaptive integration grounded in disciplined inquiry may be a critical professional skill that 
enables educators to take a productive stance to managing implementation challenges (Cohen- 
Vogel, et al., 2015; Siskin, 2016). But harnessing this capacity for supporting intervention 
scaling requires that we develop a learning infrastructure including feedback routines that enable 


research-practice partnerships to identify and spread productive adaptations. 
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In conclusion, our continuous improvement research approach to designing a scalabe 
instructional coaching model led us to important insights about mutual adaptation of the 
intervention during implementation. Our experience points to the need to systematically monitor 
practice adaptation during implementation. Studying adaptation can help researchers and 
designers to identify the essential core features of an intervention, spot potentially lethal 
mutations, and even identify productive adaptation that should be incorporated into the 
intervention design. Building educator capacity for principled adaptation that maintains the 
integrity of an intervention is a critical form of professional practice in this reform era that can be 
facilitated through improvement research methods. Together these findings provide a potential 
process model for an evidence-based approach to mutual adaptation that can be explored in 


future design-based research-practice partnerships. 
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Notes 


The Coaching Model built on the practical knowledge accumulated by the Institute for 
Learning (IFL), an outreach of the University of Pittsburgh's Learning Research and 
Development Center (LRDC). Fellows from the IFL worked in collaboration with Tennessee 
Department of Education staff and LRDC researchers to iteratively refine and test the 
coaching model that ultimately became the TN + IFL Math Coaching Model. 

Scoring of the videos led to a construct measuring students’ opportunities to engage in 
conceptual thinking during the lesson. 

Our partners from the TN DOE distributed an announcement to all school districts. Interested 
coaches submitted a written application, which included statements of interest and experience 
and effectiveness as a coach, and a performance task including identification and analysis of 
a high-level math task. We received 62 applications. Applicants then participated in a 
performance-based oral interview, which included analysis of two scenarios of mathematics 
instruction and role playing coaching interactions. We scored each component of the process 
and selected 32 coaches that scored relatively higher, but also represented variation in prior 
experience and training, and coaching context (e.g., urban, suburban, and rural. 

In this model, we treat the coach-teacher pairs as independent even though each coach 
worked with two partner teachers. In addition to the models we present, we also ran four- 
level models where the coach-teacher pairs were nested in coaches. The parameter estimates 
were roughly the same. Given the sparseness in our data, i.e., two teachers per coach, with 
some teachers missing data on the partner-teacher surveys, we chose to go with the three- 
level model. Additionally, if coaches do adapt their coaching based on the teacher, then it 
could be argued theoretically that each coach-teacher interaction represents an independent 
observation. 

We also examined models with additional proxies such as grade level and the cognitive 
demand of the task as-written for each cycle. These models did not change the parameters 
and were ultimately eliminated for model parsimony. 

Simultaneously, the model estimates of reliability for between-teacher differences in slopes 
(or linear change over time), were lower, likely due to a combination of a limited number of 
items per construct, a limited number of time points per teacher and a low ICC. However, 
because items were primarily chosen to theoretically represent the constructs explicitness and 
press, respectively, we treat this as a potential limitation of our analyses. If anything, the low 
reliability suggests we will have a more difficult time identifying covariates that predict 
variance in those linear slopes. 

While there is an association between higher teacher responsiveness and explicitness at Cycle 
1 (B201 in Table 1), teacher responsiveness does not predict status in subsequent models with 
time centered differently. 

Although the trajectories look slightly different initially, the estimated curves look similar 
over time. There are no significant differences in the model-based linear (B11 = -.54, s.e. = 
.82, p = .508), quadratic (B21 = 1.59, s.e. = 2.18, p = .468) or cubic (B31 = -.92, s.e. = 1.41, p = 
.515) growth for the two groups of teachers. 

Plan, Do, Study, Act cycles are a structured inquiry routine used to test a change idea, such 
as a new practice or tool, or a modification to an existing work process (Langley, et al., 
2009). Each PDSA is, in essence, a mini-experiment involving the introduction of some 
change, a prediction of what the outcome will be, and the collection of evidence to examine 
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whether the change appears to be an improvement (Hannan, et al., 2017). The cycle 
concludes with analysis and reflection, leading to identification of next steps, and often 
another inquiry cycle. 
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Appendix A 


Before running the models, item sub-scores from the partner-teacher survey were standardized to 
have a mean of zero and standard deviation of one (see, e.g., Raudenbush, Rowan, & Kang, 
1991). The psychometric phase of these models allowed us to examine the multivariate outcomes 
and properties of the estimable variance between teachers. The analytic phase of these models 
also allowed us to examine differences in teacher perceptions of change on the press and 
explicitness scales in addition to examining which covariates predict higher scores at Cycle 1, as 
well as higher growth slopes over the course of the three cycles. 


The level 1 model is: 
Item Scoremy = wiyj*(Pressmij) + Woj*(Explicitnessmij) + Emij [equation 2.1] 


Where Item Score is the Z-scored item for scale m at time i for teacher-coach cycle j; Pressmi 
is a dummy indicator demarcating two items for the scale Press; yj; is the average score on the 
Press scale at time i for teacher-coach pair j; Explicitnessmi is a dummy indicator demarcating 
three items for the scale Coach Explicitness; yi is the average score on the Explicitness scale at 
time i for teacher-coach pair j; émj is the measurement error for dimension m at time i for 
teacher-coach pair J. 


The level 2 model is written as follows: 
Wij = 10 + T11j*(Time) + mipj*(Api)+ eri 
[equation 2.2] 
Wij = M2oj + 121;*(Time) + mipj*(Api)t e2i 


Where 770; is the Press scale score at time | (because time is centered at the first cycle) for 
teacher-coach pair j; Time is coded as one-third at Cycle 1 and 1 at Cycle 3; 277; is the growth 
slope, in years, for the Press scale; Ap; is a set of (p) time-varying covariates measured at each 
time point; z7p; is the effect of time-varying covariates on the Press scale at each time point; e7; is 
residual error normally distributed with mean of 0 and standard deviation of unity; 729; is the 
Explicitness scale score at time | (because time is centered at the first cycle) for teacher-coach 
pair j; Time is coded as one-third at Cycle 1 and 1 at Cycle 3; 22); is the growth slope, in years, 
for the Explicitness scale; Ap; is a set of (p) time-varying covariates measured at each time point; 
72pj 1S the effect of time-varying covariates on the Explicitness scale at each time point; e2; is 
residual error normally distributed with mean of 0 and standard deviation of unity. 


The level 3 model is written as: 


Q 
110; = B100 + & Bog Xq + F109; 
q=1 
Q 
mij = Brio +X Brig Xo trig 
q=l [equation 2.3] 
Q 


720; = f200 + & B20q Xq + 120; 
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q=1 
Q 

721; = B210 + & Brig Xg t+ ray 
q=1 


Where /700 is the average Press scale score at Cycle | across all teacher-coach pairs; X, is a set 
of (q) teacher-coach covariates; io, is the effect of teacher-coach covariates on the Press scale 
score at Cycle 1; rio; is residual error normally distributed with a mean of 0 and a standard 
deviation of unity; £770 is the average linear growth in the Press scale over the year across all 
teacher-coach pairs; Xz, is a set of (q) teacher-coach covariates; £71, is the effect of teacher-coach 
covariates on linear growth in the Press scale over the year; r11; is residual error normally 
distributed with a mean of 0 and a standard deviation of unity; £200 is the average Explicitness 
scale score at Cycle | across all teacher-coach pairs; Xy is a set of (q) teacher-coach covariates; 
f20q 18 the effect of teacher-coach covariates on the Explicitness scale at Cycle 1; 120; is residual 
error normally distributed with a mean of 0 and a standard deviation of unity; £270 is the average 
linear growth in the Explicitness scale over the year across all teacher-coach pairs; Xq is a set of 
(q) teacher-coach covariates; /21, 1s the effect of teacher-coach covariates on linear growth in the 
Explicitness scale over the year; r21; is residual error normally distributed with a mean of 0 and a 
standard deviation of unity. 


Learning from Adaptation 55 


Tables 
Table 1. Effects of coach judgments on partner teacher perceptions of change in coaching 
interactions 


Partner Teacher Report of Press for 
Mathematical Ideas/ Reasoning 
Coeff. S.e. Coeff. S.e. 

Alt First Cycle 

Average (B100) 01 18 00 18 

Tch. Resp. (Bio1)# -.06 27 -.06 28 

PC Effency (Broz) 09 52 

Growth Slope 

Average (Br1o) -.03 al -.06 a, 

Tch. Resp. (B111)# A8 wif A8 | ~ .26 

PC Effency (Bi12) -.50 56 
Task Length (7102) 01 00 O1 | * 00 

Partner Teacher Report of 
Coach Explicitness 
Coeff. S.e. Coeff. S.e. 

Alt First Cycle 

Average (B200) -.06 12 -.06 11 

Tch. Resp. (B201)# 231. /° 16 31 15 

PC Effency (Booz) 31 -.88 i, 29 

Growth Slope 

Average (B210) 05 16 .06 15 

Tch. Resp. (B211) # -.55 | 20 -.54 | 19 

PC Effency (B212) 51 1.16 | 48 
Task Length (As 01 .00 O01 00 
Implemented) 


——— SSS 
+ Coaches’ reports of the likelihood that teacher follows through on the reported interaction in coach tracker during 

all reported interactions during the first cycle. 

# Efficiency of the pre-conference rigor score. The sum of items on the pre-conference rubric divided by the length 
of time in pre-conference. More efficient ratings tended to be short pre-conferences. 


Table 2. Adaptive Integration Groups and Distribution of Coaches 


# of Coaches 
Supporting beginning teachers 8 
Supporting veteran teachers 6 
Managing time to enable intensive coaching interaction 6 
Coaching teachers to attend to student conceptual 
understanding in more procedurally-focused contexts 6 


Adapting the coaching model for use in literacy 6 


Figures 


Figure 1. TN + IFL Math Coaching Model 
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Student Conceptual 
Understanding of 
Mathematics & 
Mathematical 
Performance 
Consistent with the 
STATE Standards 


Figure 2. Coach-Teacher Discussion Process 
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@ Goal and then Task Selection 


Coach & teacher set or 


clarify the mathematical 
learning goals 


(2) Pre-Observation Planning 
Conference (20-45 min.) 


Coach & teacher schedule 
pre-observation planning 


conference within 24-48 
hours before lesson 


(3) Lesson Observation 


Coach observes the teacher 


teaching the lesson 


(4) Post Observation 
Conference (20-45 min.) 


Coach & teacher schedule 


post-observation conference 
within 48 hours of observation 


Coach & teacher communicate 
to select a high level task for 
the cycle 


Coach & teacher independently 
work out solutions for task prior 
to planning conference 


Teacher is asked 
to commit to 
enacting in class 
what has been 
discussed 
(a Call to Action) 


Coach & teacher engage ina 
deep & specific discussion of 
the mathematical goals and 
the pedagogy to support 
student learning 
(the Instructional Triangle) 


Coach & teacher mark specific 
pedagogical goals in service of 
the the mathematical goal for 
the lesson and both commit to 
working toward the goals 


Coach & teacher gather evidence related to student 
understanding of the mathematical goals & 


Strategic 
and limited 
coach assist 


pedagogy that supports student learning 


Coach & teacher analyze 
evidence to highlight the goals 
that were and were not 
accomplished via 
evidence-based feedback 


Coach & teacher co-construct 
pedagogical goals for 
next cycle and commit 
to working toward goals 


